A semiconductor memory device having a three-dimensional structure has been proposed, in which memory holes are formed in a stacked body including a plurality of electrode layers stacked via an insulating layer, and a silicon body serving as a channel is provided on a side wall of the memory hole via a charge storage layer. The electrode layers function as control gates in memory cells, and integration of the semiconductor memory device is increased by reduction of the sizes of the memory holes and increase of the number of the memory holes. However, with shrink of the memory cells, interconnects connected via the memory holes may become dense and a defect may occur in memory operation.